{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Towards Visually Explaining Variational Autoencoders\n",
    "To run this notebook, activate the conda environment via the environment.yml file and start jupyter notebook within this environmnent. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pretrained networks for the different models are available at:\n",
    "\n",
    "https://drive.google.com/drive/folders/1OrX5HuH6vjSD8D2gLLv7ANc1iBUuVGHY?usp=sharing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cd Anomaly_Detection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MNIST Anomaly Detection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "run -t code/test_expVAE.py --dataset=mnist --model=vanilla_mnist --batch_size=32 --model_path='/media/bob/OS/Users/boble/Documents/AI - year 1/FACT-AI/MNIST_model/MNIST_model/7/model_best.pth'  --target_layer='encoder.2'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "run -t code/show_results.py --dataset=mnist --model=vanilla_mnist "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "img = random.choice([x for x in os.listdir(\"test_results/vanilla\")\n",
    "               if os.path.isfile(os.path.join(\"test_results/vanilla\", x))], )\n",
    "print(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## USCD-Ped1 Anomaly Detection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Info:**  training one model can take up to 1 hour on a gpu. Downloading the dataset takes up 5 Gb."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To train the authors configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run -t code/train_expVAE.py --model vanilla_ped1 --dataset ucsd_ped1 --batch_size 32 --epochs 512 --batch_norm False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To test the authors configuration:\n",
    "Make sure you download the model first and put it in the ```./ckpt``` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run -t code/test_expVAE.py --model vanilla_ped1 --dataset ucsd_ped1 --batch_size 4 --model_path './ckpt/vanilla_ped1_best_original.pth' --target_layer encoder.5 --image_size 100 --batch_size False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To test the our best configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run -t code/test_expVAE.py --model vanilla_ped1 --dataset ucsd_ped1 --batch_size 4 --model_path './ckpt/vanilla_ped1_best_our_best.pth' --target_layer encoder.8 --image_size 96 --batch_norm True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To compute the VAE reconstruction baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run -t code/test_vanVAE.py --image_size 96 --reconstruction True --model vanilla_ped1 --dataset ucsd_ped1 --batch_size 4 --model_path './ckpt/vanilla_ped1_best_our_best.pth' --image_size 96"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To compute the new Average VAE reconstruction baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run -t code/test_vanVAE.py --image_size 96 --reconstruction False --model vanilla_ped1 --dataset ucsd_ped1 --batch_size 4 --model_path './ckpt/vanilla_ped1_best_our_best.pth' --image_size 96"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MVTec Anomaly Detection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To train the models:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Info:**  training one model can take up to 1 hour on a gpu. Downloading the dataset takes up 5 Gb."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Specify the target as the index according to the following list(e.g. bottle: 0, cable: 1): \n",
    "\n",
    "['bottle', 'cable', 'capsule', 'carpet', 'grid',\n",
    "               'hazelnut', 'leather', 'metal_nut', 'pill', 'screw',\n",
    "               'tile', 'toothbrush', 'transistor', 'wood', 'zipper']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "object_class = 5 # 5 is nut\n",
    "pre_trained_model= './ckpt/resnet18_3_mvtecClass_'+str(object_class) +'_final.pth'\n",
    "self_trained_model = './ckpt/resnet18_3_mvtecClass_'+str(object_class) +'_checkpoint.pth'\n",
    "target_layer = 'encoder.layer2.1.conv1'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "run -t  code/train_expVAE.py --dataset=mvtec_ad --model=resnet18_3 --batch_size=8 --one_class=$object_class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To test the self-trained model:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Info:** to find the best IoU score set the flag: --iou True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "run code/test_expVAE.py --dataset=mvtec_ad --model=resnet18_3 --batch_size=2 --model_path=$self_trained_model --one_class=$object_class --target_layer=$target_layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To load pretrained models:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "run -t code/test_expVAE.py --dataset=mvtec_ad --model=resnet18_3 --batch_size=2 --model_path=$pre_trained_model --one_class=$object_class --target_layer=$target_layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run -t code/show_results.py --dataset=mvtec_ad --model=resnet18_3 --one_class=$object_class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## dSprites Latent Space Disentanglement\n",
    "Make sure you are in the Latent_Space_Disentanglement directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cd ../Latent_Space_Disentanglement/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have not yet downloaded the dSprites dataset, uncomment the following cell and run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !sh scripts/prepare_data.sh dsprites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing attention maps\n",
    "First we reproduce the attention maps of the two highest response latent dimensions for the baseline FactorVAE and the best performing AD-FactorVAE using $\\lambda=40$ and the first convolutional layer. Press any key to close the image. (Using the ```--help``` flag you can find more options)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python visualizer.py --name FactorVAE --target_layer 0 --cuda --sample_count 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python visualizer.py --name AD-FactorVAE --target_layer 0 --cuda --sample_count 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting the results\n",
    "Next we plot the results for all the models used, corresponding to 3a in the report. Using the ```--help``` flag you can find more options, for example ```--all_plots``` will show more information about the loss progression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%run plotter.py --names 'gamma40 lambda1_gamma40 lambda20_gamma40 lambda40_gamma40 conv3_lambda1_gamma40'"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
